{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1.使用核函数的原因\n",
    "特征空间维数可能很高，直接计算有困难\n",
    "\n",
    "2.1 支持向量\n",
    "在支持向量机中,距离超平面最近的且满足一定条件的几个训练样本点。\n",
    "\n",
    "2.2为什么SVM中只有一小部分的训练样本是支持向量（稀疏的）？\n",
    "只有超平面附近的样本在分类的时候需要注意，距离超平面较远的样本点不会对分类的结果造成影响，只考虑超平面附近的样本点分类结果比较精准计算量也大大减小。\n",
    "\n",
    "3.决策树中特征分裂的准则\n",
    "对连续属性进行升序排序，只有在决策属性（即分类发生了变化）发生改变的地方才需要切开，这可以显著减少运算量。\n",
    "（1） 对特征的取值进行升序排序\n",
    "（2） 两个特征取值之间的中点作为可能的分裂点，将数据集分成两\n",
    "部分，计算每个可能的分裂点的Gini指数。\n",
    "（3）选择Gini指数最小的分裂点作为该特征的最佳分裂点。\n",
    "\n",
    "4.SVM模型并没有概率解释。为了使SVM模型能输出概率，我们应该设置哪个参数？\n",
    "参数probability被设为True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "#用正确率作为模型的度量\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "#把结果用文本的形式打印出来\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#读取数据\n",
    "#采用原始特征+tf_idf特征\n",
    "#原始特征+tf_idf特征对线性SVM训练很快，RBF核函数慢\n",
    "train1 = pd.read_csv(\"Otto_FE_train_org.csv\")\n",
    "train2 = pd.read_csv(\"Otto_FE_train_tfidf.csv\")\n",
    "\n",
    "train2 = train2.drop([\"Target\"], axis=1)\n",
    "train =  pd.concat([train1, train2], axis = 1, ignore_index=False)\n",
    "train.head()\n",
    "\n",
    "del train1\n",
    "del train2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将类别字符串变成数字\n",
    "# drop ids and get labels\n",
    "y_train = train['Target']   #形式为Class_x\n",
    "X_train = train.drop([ \"Target\"], axis=1)\n",
    "\n",
    "#保存特征名字以备后用（可视化）\n",
    "feat_names = X_train.columns \n",
    "\n",
    "#sklearn的学习器大多之一稀疏数据输入，模型训练会快很多\n",
    "from scipy.sparse import csr_matrix\n",
    "X_train = csr_matrix(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E:\\Anaconda\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "#将矩阵随机划分为训练子集和测试子集，并返回划分好的训练集测试集样本和训练集测试集标签\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train_part, X_val, y_train_part, y_val = train_test_split(X_train, y_train, train_size = 0.75,random_state = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
       "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
       "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
       "     verbose=0)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "#LinearSVC不能得到每类的概率（只有predict函数，没有predict_proba函数），在Otto数据集要求输出每类的概率，这里只是示意SVM的使用方法\n",
    "#https://xacecask2.gitbooks.io/scikit-learn-user-guide-chinese-version/content/sec1.4.html\n",
    "#1.4.1.2. 得分与概率\n",
    "#1. 生成学习器实例\n",
    "SVC1 = LinearSVC()\n",
    "\n",
    "#2. 模型训练\n",
    "SVC1.fit(X_train_part, y_train_part)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is: [0.75974026 0.75324675 0.76623377 0.81699346 0.75163399]\n",
      "cv logloss is: 0.7695696460402344\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "loss=cross_val_score(SVC1,X_train,y_train,cv=5,scoring='accuracy')\n",
    "print('logloss of each fold is:',loss)\n",
    "print('cv logloss is:',loss.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy is:  0.8020833333333334\n",
      "Classification report for classifier LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
      "     verbose=0):\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.82      0.90      0.86       130\n",
      "          1       0.74      0.60      0.66        62\n",
      "\n",
      "avg / total       0.80      0.80      0.80       192\n",
      "\n",
      "\n",
      "Confusion matrix:\n",
      "[[117  13]\n",
      " [ 25  37]]\n"
     ]
    }
   ],
   "source": [
    "#3. 在校验集上测试，估计模型性能\n",
    "y_predict = SVC1.predict(X_val)\n",
    "\n",
    "print(\"accuracy is: \",accuracy_score(y_val, y_predict))\n",
    "\n",
    "print(\"Classification report for classifier %s:\\n%s\\n\"\n",
    "      % (SVC1, classification_report(y_val, y_predict)))\n",
    "print(\"Confusion matrix:\\n%s\" % confusion_matrix(y_val, y_predict))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "#单组超参数情况，模型在训练集上训练，在校验集上的测试的测试性能\n",
    "def fit_grid_point_Linear(C, X_train, y_train, X_val, y_val):\n",
    "    \n",
    "    # 在训练集上训练SVC\n",
    "    SVC2 =  LinearSVC( C = C)\n",
    "    SVC2 = SVC2.fit(X_train, y_train)\n",
    "    \n",
    "    # 在校验集上返回accuracy\n",
    "    accuracy = SVC2.score(X_val, y_val)\n",
    "    \n",
    "    print(\"C= {} : accuracy= {} \" .format(C, accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 0.1 : accuracy= 0.796875 \n",
      "C= 0.1 : accuracy= 0.796875 \n",
      "C= 1.0 : accuracy= 0.8020833333333334 \n",
      "C= 1.0 : accuracy= 0.8020833333333334 \n",
      "C= 10.0 : accuracy= 0.8020833333333334 \n",
      "C= 10.0 : accuracy= 0.8020833333333334 \n",
      "C= 100.0 : accuracy= 0.7864583333333334 \n",
      "C= 100.0 : accuracy= 0.8177083333333334 \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No handles with labels found to put in legend.\n",
      "No handles with labels found to put in legend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 1000.0 : accuracy= 0.75 \n",
      "C= 1000.0 : accuracy= 0.4010416666666667 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHqpJREFUeJzt3XmYXGWZ/vHv091pmhAgJJ0gZgcCkoAEaFnEcVjEwWUScUHWH6CADAaQYYCwhoQlQUe5YAQ0gwswDougEhHIRBYRBKTZs7AkYWuDEAIhIFlI+vn98Z5qik53V3V3nXpPVd+f6zrX6ap+q+quSvo8Ve+p8xxzd0RERABqYgcQEZHsUFEQEZE2KgoiItJGRUFERNqoKIiISBsVBRERaaOiICIibVQURESkjYqCiIi0qYsdoLsaGxt99OjRsWOIiFSUxx577E13H1JoXMUVhdGjR9Pc3Bw7hohIRTGzl4sZp+kjERFpo6IgIiJtVBRERKRNxe1TEBHp6z744ANaWlpYvXr1Br9raGhg+PDh9OvXr0f3raIgIlJhWlpa2HTTTRk9ejRm1na9u7N8+XJaWloYM2ZMj+5b00ciIhVm9erVDB48+CMFAcDMGDx4cIefIIqloiAiUoHaF4RC1xdL00cikrqVK2H+fFiwABYtgpdeghUr4Gc/g49/PHY6yaeiICI99ve/wzPPwLPPho39K6/A0qXw5ptho/+Pf8DatdDZqeC/9S24667yZpauqSiIyEe0tsLixTBvHjz3XHhX/8oroQC8+WZ41//++/DBB53fR20tNDTAFlvAoEGw5ZYwfDiMHg3bbQfjxsG++8I994THq9FEdre5e4dTRd5ZBS6SioJIH7F2bZi+mT8fXngBXnwRWlrg9dfhrbfCxn71ali/vvP76NcPNt4YPvYxGDw4rEeMgK23hu23hx13hG22KW4j/41vhOmjq66CyZNL9zz7goaGBpYvX77Bzubct48aGhp6fN/W26pSbk1NTa7eRyIfWrkSnn76o/P1S5fCG2+EKZz33oM1a8I78o6YQX09bLIJDBwIjY2w1VYwcmTYwH/iE7DTTqWf+3/jjfAJYtttQ5GS4vXkOAUze8zdmwrdtz4piGTU0qUfztcvXrzhfP3773c9X19TAxttBAMGhHfzQ4fCsGEwahSMHQs77BA29pttVt7nlTN0aCg4zz4bPrEMHx4nRyXq169fj49DKCTVomBmBwKXA7XANe4+s93vRwLXAgOTMVPc/Y40M4nE1Noa3hXPmwfPPw9LlsCrr4b5+uXLP5yvX7eu8/uorQ1TOIMGhSmc3Hz9mDFhvn78+LCxra8v3/PqqdNOg+OOg3POgWuvjZ1GIMXpIzOrBZ4HDgBagEeBQ919Qd6YWcAT7n61mY0D7nD30V3dr6aP0rVuXdgwvfMOvPtu7DSVZdWqsMF//vkwhZM/X//uu8XN1/fvD5tvHqZwOpqvHzOmunbKtraG59yvn/6/pS0L00e7A4vcfUkS6EZgErAgb4wDuQ+vmwNLU8xTMVpbwzzw22+HjfPKlWG64N13P1zeey983S+3vP9+2CitWhU2PqtXh3nkNWvCFMPatWGD/8EHYb1+/YeLe+fzzdI7ZmEKp3//8I5+yJAwNz9qVJivz03hDB0aO2kcNTVwwAFw++0wezZMnBg7kaRZFIYBr+ZdbgH2aDfmAuD/zOwkYBPgcynm6ZbW1rChXbEibJhXrAgb5442zLn1qlUfbpxXrw7r3EZ5zZqwQc4t69aFpbU1bJhbW8PGuZQf3MzCUlsb/vhqa6GuLkwr9Ov30XVDQ9h4NTSEqYmNNw6Xe3lwZJ9SVxemccaODV+5HD8+zOdL12bODEVh+nQVhSxIsyh0tDlpv8k7FPilu//QzPYCrjezHd39I+9bzex44HiAkSNH9ijMtGlw3XVhA91+w5x7x5zmhjm3Uc4tdXVhw5vbKNfXh41w+w3zxhuHb4X07x/WAwZ8uGy+OWy6adhROHBguDxwYBgrUinGjw/fdnr88fAGS4U0rjSLQgswIu/ycDacHvo2cCCAuz9kZg1AI/BG/iB3nwXMgrBPoSdhnn8+fHsjf+NcVxc2tP36hSW3Ua6vDxvjhoawbLJJuNy/f/gPm79xbr9hzq0HDKiuuV+RNB1/fHjjduGFcOmlsdP0bWnuaK4j7GjeH/gbYUfzYe4+P2/MncBN7v5LM9sBuBsY5l2E0o5mkeqzenV4s9XYGHbOS+kVu6M5tfey7r4OmAzMARYCN7v7fDObbma5mcPTgOPM7CngBuDorgqCiFSnhgbYY49wQNvjj8dO07fpiGYRyYS5c+Hznw/LnDmx01Sf6J8URES644ADwj66e+/VV6RjUlEQkcw4+ODwzcArr4ydpO9SURCRzLjoorC+/PK4OfoyFQURyYxck7zFi0ObECk/FQURyZT/+I+wPvvsuDn6KhUFEcmUY44JB5H+5jexk/RNKgoikik1NfAv/xL6id12W+w0fY+KgohkzszkzCsXXhg3R1+koiAimbPDDqHFeK5JnpSPioKIZNLxx4eOxdOnx07St6goiEgmnXlm2L/wy1/GTtK3qCiISCY1NMCee8KyZaB2Z+WjoiAimZWbOtIxC+WjoiAimbX//mqSV24qCiKSaYccEk6b+1//FTtJ36CiICKZlmuSd8UVcXP0FSoKIpJpjY3huIUlS8J51iVdKgoiknmnnx7W2uGcPhUFEcm8o44KTfJ+97vYSaqfioKIZF5NDRx4YGiS99vfxk5T3VQURKQizJgR1rkdz5IOFQURqQg77ADDhsETT6hJXppUFESkYuSa5E2bFjtJ9VJREJGKMWVK2L9w7bWxk1QvFQURqRj19bDXXqFJ3qOPxk5TnVQURKSiqEleulQURKSi7LcfDBwI990XeiJJaakoiEjFyTXJ+/GPYyepPioKIlJxLrwwrNU5tfRUFESk4jQ2wrhxapKXBhUFEalIZ5wR1lOmxM1RbVQURKQiHXlkOI/z7Nmxk1QXFQURqUj5TfJuvTV2muqhoiAiFWvmzLC++OK4OaqJioKIVKzttw9N8p58ElaujJ2mOqgoiEhFO+EENckrJRUFEaloZ5wBtbVw3XWxk1QHFQURqWi5JnlvvgmPPBI7TeVLtSiY2YFm9pyZLTKzDb5NbGaXmdmTyfK8ma1IM4+IVKfcEc7nnBM3RzWoS+uOzawWuBI4AGgBHjWz2e6+IDfG3U/NG38SsEtaeUSkeu2zT2iS96c/hZ5Idalt2apfmp8UdgcWufsSd18L3AhM6mL8ocANKeYRkSp26KGhIFxxRewklS3NojAMeDXvckty3QbMbBQwBrgnxTwiUsUuuiis1Tm1d9IsCtbBdd7J2EOAW9x9fYd3ZHa8mTWbWfOyZctKFlBEqsegQTB+PLz4Irz8cuw0lSvNotACjMi7PBxY2snYQ+hi6sjdZ7l7k7s3DRkypIQRRaSaqEle76VZFB4FxprZGDOrJ2z4N2hdZWbbA1sAD6WYRUT6gCOOUJO83kqtKLj7OmAyMAdYCNzs7vPNbLqZTcwbeihwo7t3NrUkIlKUmhr4whfg/ffhlltip6lMVmnb4qamJm9ubo4dQ0Qy6oUXYLvtYMIEeOKJ2Gmyw8wec/emQuN0RLOIVJWxY2HECHjqKTXJ6wkVBRGpOt/5TmiSd8EFsZNUHhUFEak6p5+uJnk9paIgIlWnvh4+/WlYvlxN8rpLRUFEqlLuCOezz46bo9KoKIhIVfrsZ2GLLeD++0NPJCmOioKIVK1ck7zLL4+dpHKoKIhI1cqdZ0FN8oqnoiAiVWvQINhxR3jpJTXJK5aKgohUtTPP/OhauqaiICJV7bDDQpO83/8+dpLKoKIgIlWtpga+9KXQJO/mm2OnyT4VBRGpejNmhPUll8TNUQlUFESk6uWa5D39tJrkFaKiICJ9wr/9W2iSN3Vq7CTZpqIgIn3CaaeFJnnXXx87SbapKIhIn1BfD3vvHZrkPaST/3ZKRUFE+oyLLw7rc86JmyPLiioKZnarmX3JzFRERKRifeYzoUnen/+sJnmdKXYjfzVwGPCCmc00s0+kmElEJDWHHx4KwmWXxU6STUUVBXf/o7sfDuwKvATMNbO/mNkxZtYvzYAiIqU0bRqYwZVXxk6STUVPB5nZYOBo4FjgCeByQpGYm0oyEZEU5JrkvfwyvPhi7DTZU+w+hd8Afwb6A//q7hPd/SZ3PwkYkGZAEZFSO+ussFaTvA2ZuxceZLafu99ThjwFNTU1eXNzc+wYIlLh+vcP6/ffj5ujXMzsMXdvKjSu2OmjHcxsYN6db2FmJ/Y4nYhIZF/6EqxaBTfdFDtJthRbFI5z9xW5C+7+NnBcOpFERNI3c2ZYq0neRxVbFGrMzHIXzKwWqE8nkohI+rbZBkaOhGeegRUrCo/vK4otCnOAm81sfzPbD7gBuCu9WCIi6VOTvA0Vu6O5BvgOsD9gwP8B17j7+nTjbUg7mkWkVNatC2dl23zz0BOpmpV0R7O7t7r71e7+dXf/mrv/NEZBEBEppbq60PrirbfgwQdjp8mGYo9TGGtmt5jZAjNbklvSDicikrZck7xzz42bIyuK3afwC0L/o3XAvsB1gLqSi0jF23vvcJSzmuQFxRaFjd39bsI+iJfd/QJgv/RiiYiUz+GHw/r18KMfxU4SX7FFYXWys/kFM5tsZgcBQ1PMJSJSNtOnq0leTrFF4XuEvkcnA7sBRwBHpRVKRKScBg4MTfJeeQUWL46dJq6CRSE5UO1gd3/P3Vvc/ZjkG0gPlyGfiEhZ5M7GNmVK3ByxFSwKyVdPd8s/ollEpNp885uw8cbwhz/EThJXsdNHTwC3mdmRZvbV3JJmMBGRcvvyl0OTvBtuiJ0knmKLwiBgOeEbR/+aLF9OK5SISAyXXhrWM2bEzRFTXTGD3P2Ynty5mR1IOENbLaEtxswOxhwMXAA48JS7H9aTxxIR6a0xY2DUKJg3LzTJGziw8G2qTVFFwcx+Qdhof4S7f6uL29QCVwIHAC3Ao2Y2290X5I0ZC5wF7O3ub5uZvuYqIlGdeGI4I9v558MVV8ROU37FTh/dDvwhWe4GNgPeK3Cb3YFF7r7E3dcCNwKT2o05DrgyOT8D7v5GscFFRNLw7/8OtbXwq1/FThJHsQ3xbs1bfgUcDOxY4GbDgFfzLrck1+XbDtjOzB40s4eT6SYRkWjq6uCf/ik0yXvggdhpyq/YTwrtjQVGFhjT0VdY209B1SX3tQ9wKHBN/mk/2+7I7Hgzazaz5mXLlvUgrohI8S66KKz7YpO8YrukvmtmK3ML8HvgzAI3awFG5F0eDiztYMxt7v6Bu78IPEcoEh/h7rPcvcndm4YMGVJMZBGRHss1yXvgAVi7Nnaa8ip2+mhTd98sb9nO3W8tcLNHgbFmNsbM6oFDgNntxvyO0HUVM2skTCepJbeIRHfEEX2zSV6xnxQOMrPN8y4PNLOvdHUbd18HTCacynMhcLO7zzez6WY2MRk2B1huZguAe4HT3b3Kz38kIpXgwgtDk7yrroqdpLyKPR3nk+4+od11T7j7Lqkl64ROxyki5bLzzvD007BoEWyzTew0vVPS03F2Mq6oYxxERCpVrknemYX2oFaRYotCs5n9yMy2MbOtzewy4LE0g4mIxHbwwaFJ3h13xE5SPsUWhZOAtcBNwM3AKuC7aYUSEcmKiRNDk7y+cjBbUfsUskT7FESknF58EbbeGnbaKexfqFQl3adgZnPzDyozsy3MbE5vAoqIVIL8JnlvvRU7TfqKnT5qdPcVuQtJryI1rxORPuG73wV3mDo1dpL0FVsUWs2sra2FmY2mg66pIiLV6NRTQ0+kvrBfodiicA7wgJldb2bXA38itLwWEal6uSZ5b79d/U3yim1zcRfQROhNdBNwGuEbSCIifcLFF4d17tiFalXsSXaOBU4hNLV7EtgTeIhwek4Rkaq3114weDA8+GBokldfHztROoqdPjoF+BTwsrvvC+wCqIe1iPQpRx4ZmuT98Iexk6Sn2KKw2t1XA5jZRu7+LLB9erFERLJn2rTQJO/qq2MnSU+xRaElOU7hd8BcM7uNDc+NICJS1TbbDD75SXj1VXjhhdhp0lHsjuaD3H2Fu18AnAf8DOiydbaISDU6++ywnjIlbo60qM2FiEg3bbIJtLbCP/4BNT09qXGZlbp1toiIJCZOhNWr4X//N3aS0lNREBHpppkzw/rSS+PmSIOKgohIN40aBaNHV2eTPBUFEZEemDw5rM87L26OUlNREBHpgVNOCT2RbrghdpLSUlEQEemBujr4538OTfLuvz92mtJRURAR6aFck7xzz42bo5RUFEREemiPPaCxEf7yl9AkrxqoKIiI9EKuSd4PfhA7SWmoKIiI9MIFF4QmeT/5SewkpaGiICLSC5ttBjvvDC0t8NxzsdP0noqCiEgv5XY0n1UFJylWURAR6aWvfQ3694c77wyN8iqZioKISAlMmhSa5P3P/8RO0jsqCiIiJTBjRlh///txc/SWioKISAmMGgVjxsD8+ZXdJE9FQUSkRE4+Oawr+QhnFQURkRKZPLnym+SpKIiIlEhdHeyzD6xYAffcEztNz6goiIiU0CWXhPXUqXFz9JSKgohICX3qUzBkCDz0UGU2yVNREBEpsVyTvEr8eqqKgohIiU2bVrlN8lItCmZ2oJk9Z2aLzGxKB78/2syWmdmTyXJsmnlERMphwACYMAH+9rfKa5KXWlEws1rgSuALwDjgUDMb18HQm9x9QrJck1YeEZFyOu+8sJ6ywdvhbEvzk8LuwCJ3X+Lua4EbgUkpPp6ISGYcdBBssgncdVdlNclLsygMA17Nu9ySXNfe18zsaTO7xcxGpJhHRKSsck3yrr8+dpLipVkUrIPrvN3l3wOj3f2TwB+Bazu8I7PjzazZzJqXLVtW4pgiIumoxCZ5aRaFFiD/nf9wYGn+AHdf7u5rkov/DezW0R25+yx3b3L3piFDhqQSVkSk1EaOhK23hgUL4M03Y6cpTppF4VFgrJmNMbN64BBgdv4AM9sq7+JEYGGKeUREyi7XJC+34znrUisK7r4OmAzMIWzsb3b3+WY23cwmJsNONrP5ZvYUcDJwdFp5RERiOOmk0BPpxhtjJymOubef5s+2pqYmb25ujh1DRKRon/88zJ0Ld98N++0XJ4OZPebuTYXG6YhmEZGU5ZrknX9+3BzFUFEQEUlZU1PlNMlTURARKYOjjgoHsc2cGTtJ11QURETKYOrU0CRv1qzYSbqmoiAiUgYDBsAuu4QmeQsz/OV7FQURkTLJ7Wg+66y4ObqioiAiUiaTJmW/SZ6KgohIGX3lK7BmDVzbYae3+FQURETKKPftox/8IG6OzqgoiIiU0fDhoUnewoXZbJKnoiAiUmbf+15Yn3tu3BwdUVEQESmz734X+vXLZpM8FQURkTKrqYF994V33glN8rJERUFEJIKLLw7rrDXJU1EQEYmgqQmGDoWHHw7ncc4KFQURkUhyTfIuvTR2kg+pKIiIRHL++dlrkqeiICISyYABsOuusHRpdprkqSiIiESU29E8ZUrcHDkqCiIiEU2cGD4xzJmTjSZ5KgoiIpEddFBokveLX8ROoqIgIhLdJZeE9X/+Z9wcoKIgIhLd8OGwzTbw7LPwxhtxs6goiIhkQK5J3jnnxM2hoiAikgEnnhia5P3613FzqCiIiGRATQ3st19okjd3bsQc8R5aRETy5XY4T50aL4OKgohIRuy6a2iS98gj8ZrkqSiIiGTI0UeHg9hy53IuNxUFEZEMmTo1bpM8FQURkQzp3x922w1eew3mzy//46soiIhkTG5Hc4wmeSoKIiIZ8+UvhyZ5c+eWv0meioKISAZ99auhSd7Pf17ex1VREBHJoBkzwvqHPyzv46ooiIhk0Mc/DttuW/4meSoKIiIZdeqpYX322eV7TBUFEZGMOuGE8jfJU1EQEcmomhrYf39YuTKcrrMsj5nmnZvZgWb2nJktMrNOv3FrZl83MzezpjTziIhUmtwO5wsuKM/jpVYUzKwWuBL4AjAOONTMxnUwblPgZOCRtLKIiFSqCRNgyy3hr38tT5O8ND8p7A4scvcl7r4WuBGY1MG4C4HvA5F6AoqIZNu3vhUOYsu11k5TmkVhGPBq3uWW5Lo2ZrYLMMLdb+/qjszseDNrNrPmZcuWlT6piEiGnXsuDBkS+iKlrS7F+7YOrvO2X5rVAJcBRxe6I3efBcwCaGpq8gLDRUSqSv/+5TtWIc1PCi3AiLzLw4GleZc3BXYE7jOzl4A9gdna2SwiEk+aReFRYKyZjTGzeuAQYHbul+7+jrs3uvtodx8NPAxMdPfmFDOJiEgXUisK7r4OmAzMARYCN7v7fDObbmYT03pcERHpuTT3KeDudwB3tLvu/E7G7pNmFhERKUxHNIuISBsVBRERaaOiICIibVQURESkjblX1rFgZrYMeLmHN28E3ixhnFJRru5Rru7Lajbl6p7e5Brl7kMKDaq4otAbZtbs7pk7OE65uke5ui+r2ZSre8qRS9NHIiLSRkVBRETa9LWiMCt2gE4oV/coV/dlNZtydU/qufrUPgUREelaX/ukICIiXajqomBm3zCz+WbW2lVL7mLPJV3CXIPMbK6ZvZCst+hk3HozezJZZnc0pkR5unz+ZraRmd2U/P4RMxudVpZu5jrazJblvUbHlinXz83sDTOb18nvzcyuSHI/bWa7ZiTXPmb2Tt7r1WEfshJnGmFm95rZwuRv8ZQOxpT99SoyV9lfr+RxG8zsr2b2VJJtWgdj0vubdPeqXYAdgO2B+4CmTsbUAouBrYF64ClgXMq5vg9MSX6eAlzaybj3yvAaFXz+wInAT5KfDwFuykiuo4EfR/h/9VlgV2BeJ7//InAn4URTewKPZCTXPsDtZX6ttgJ2TX7eFHi+g3/Hsr9eReYq++uVPK4BA5Kf+xHOX79nuzGp/U1W9ScFd1/o7s8VGFbsuaRLaRJwbfLztcBXUn68rhTz/PPz3gLsb2YdnVmv3LmicPf7gbe6GDIJuM6Dh4GBZrZVBnKVnbu/5u6PJz+/S2ijP6zdsLK/XkXmiiJ5Hd5LLvZLlvY7f1P7m6zqolCkgueSTsGW7v4ahP+cwNBOxjUk56Z+2MzSKhzFPP+2MR7Ok/EOMDilPN3JBfC1ZMrhFjMb0cHvY4jxf6pYeyXTEnea2fhyPnAyxbEL4Z1vvqivVxe5INLrZWa1ZvYk8AYw1907fc1K/TeZ6vkUysHM/gh8rINfnePutxVzFx1c1+uvZHWVqxt3M9Ldl5rZ1sA9ZvaMuy/ubbZ2inn+qbxGBRTzmL8HbnD3NWZ2AuGd034p5ypGjNerGI8TWh28Z2ZfBH4HjC3HA5vZAOBW4HvuvrL9rzu4SVlerwK5or1e7r4emGBmA4HfmtmO7p6/ryi116zii4K7f66Xd1HoXNI90lUuM3vdzLZy99eSj8kdnpLb3Zcm6yVmdh/h3Uypi0Ixzz83psXM6oDNSX+aomAud1+ed/G/gUtTzlSsVP5P9Vb+Rs/d7zCzq8ys0d1T7fFjZv0IG95fuftvOhgS5fUqlCvW69Uuw4rkb/9AIL8opPY3qemjAueSTsls4Kjk56OADT7RmNkWZrZR8nMjsDewIIUsxTz//LxfB+7xZA9XigrmajfvPJEwL5wFs4H/l3yrZk/gndx0YUxm9rHcvLOZ7U74+1/e9a16/ZgG/AxY6O4/6mRY2V+vYnLFeL2SxxqSfELAzDYGPgc8225Yen+T5d6zXs4FOIhQUdcArwNzkus/DtyRN+6LhG8fLCZMO6WdazBwN/BCsh6UXN8EXJP8/GngGcK3bp4Bvp1ing2ePzAdmJj83AD8GlgE/BXYukz/foVyzQDmJ6/RvcAnypTrBuA14IPk/9e3gROAE5LfG3BlkvsZOvnmW4Rck/Ner4eBT5ch02cI0xpPA08myxdjv15F5ir765U87ieBJ5Js84Dzk+vL8jepI5pFRKSNpo9ERKSNioKIiLRRURARkTYqCiIi0kZFQURE2qgoiHTCzN4rPKrL29+SHI2OmQ0ws5+a2eKk8+X9ZrZHaZKKlE7FH9EskkVJn5xad1+SXHUN8CIw1t1bk2KxQ7SAIp3QJwWRApIjbX9gZvPM7Bkz+2ZyfU3S+mC+md1uZneY2deTmx1OcqS6mW0D7AGc6+6tEFqXuPsfYjwfka7ok4JIYV8FJgA7A43Ao2Z2P6H1yGhgJ0Kn24XAz5Pb7E04whhgPPCkhyZnIpmmTwoihX2G0I11vbu/DvwJ+FRy/a/dvdXd/05otZGzFbCs/FFFekdFQaSwzk5e0tVJTVYR+tNA6J+zs5np700yT/9JRQq7H/hmcuKTIYTTXv4VeIBwkp8aM9uScPrGnIXAtgAezoHRDEzL67o51swycSY5kXwqCiKF/ZbQsfIp4B7gjGS66FZCN9J5wE8JZ+56J7nNH/hokTiWcNKlRWb2DOH8D9HPsSDSnrqkivSCmQ3wcGauwYRPD3u7+9+TPvj3Jpe1g1kqhr59JNI7tycnRKkHLkw+QeDuq8xsKuFcuq/EDCjSHfqkICIibbRPQURE2qgoiIhIGxUFERFpo6IgIiJtVBRERKSNioKIiLT5/62nx9a+qxUIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#需要调优的参数\n",
    "#SVM太慢，每次只调一个参数（这里只调C，penalty为‘l2'）\n",
    "C_s = np.logspace(-1, 3, 5)# logspace(a,b,N)把10的a次方到10的b次方区间分成N份  \n",
    "penalty_s = ['l1','l2']\n",
    "\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "    for j, penalty in enumerate(penalty_s):\n",
    "        tmp = fit_grid_point_Linear(oneC, X_train_part, y_train_part, X_val, y_val)\n",
    "    accuracy_s.append(tmp)\n",
    "\n",
    "x_axis = np.log10(C_s)\n",
    "for j, penalty in enumerate(penalty_s):\n",
    "    plt.plot(x_axis, np.array(accuracy_s), 'b-')\n",
    "    \n",
    "    plt.legend()\n",
    "    plt.xlabel( 'log(C' )                                                                                                      \n",
    "    plt.ylabel( 'accuracy' )\n",
    "    plt.savefig('SVM_Otto.png' )\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当C不变时，正则的变化并不会引起正确率较大的变化\n",
    "C过大会引起准确率的下降"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "### 最佳超参数\n",
    "index = np.argmax(accuracy_s, axis=None)\n",
    "Best_C = C_s[ index ]\n",
    "\n",
    "print(Best_C)### 最佳超参数\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "Best_C = 10\n",
    "\n",
    "SVC5 = LinearSVC(C = Best_C)\n",
    "SVC5.fit(X_train, y_train)\n",
    "\n",
    "#保持模型，用于后续测试\n",
    "import pickle\n",
    "pickle.dump(SVC5, open(\"Otto_LinearSVC.pkl\", 'wb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RBF核SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "SVC3 = SVC()\n",
    "SVC3.fit(X_train_part, y_train_part)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is: [0.75324675 0.73376623 0.74675325 0.74509804 0.73202614]\n",
      "cv logloss is: 0.7421780833545539\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "loss=cross_val_score(SVC3,X_train,y_train,cv=5,scoring='accuracy')\n",
    "print('logloss of each fold is:',loss)\n",
    "print('cv logloss is:',loss.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy is:  0.796875\n",
      "Classification report for classifier LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
      "     verbose=0):\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.81      0.91      0.86       130\n",
      "          1       0.74      0.56      0.64        62\n",
      "\n",
      "avg / total       0.79      0.80      0.79       192\n",
      "\n",
      "\n",
      "Confusion matrix:\n",
      "[[118  12]\n",
      " [ 27  35]]\n"
     ]
    }
   ],
   "source": [
    "#3. 在校验集上测试，估计模型性能\n",
    "y_predict = SVC3.predict(X_val)\n",
    "\n",
    "print(\"accuracy is: \",accuracy_score(y_val, y_predict))\n",
    "\n",
    "print(\"Classification report for classifier %s:\\n%s\\n\"\n",
    "      % (SVC1, classification_report(y_val, y_predict)))\n",
    "print(\"Confusion matrix:\\n%s\" % confusion_matrix(y_val, y_predict))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "#单组超参数情况，模型在训练集上训练，在校验集上的测试的测试性能\n",
    "def fit_grid_point(C, X_train, y_train, X_val, y_val):\n",
    "    \n",
    "    # 在训练集上训练SVC\n",
    "    SVC4 =  SVC( C = C)\n",
    "    SVC4 = SVC4.fit(X_train, y_train)\n",
    "    \n",
    "    # 在校验集上返回accuracy\n",
    "    accuracy = SVC4.score(X_val, y_val)\n",
    "    \n",
    "    print(\"C= {} : accuracy= {} \" .format(C, accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No handles with labels found to put in legend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C= 0.1 : accuracy= 0.6770833333333334 \n",
      "C= 1.0 : accuracy= 0.7604166666666666 \n",
      "C= 10.0 : accuracy= 0.8020833333333334 \n",
      "C= 100.0 : accuracy= 0.7864583333333334 \n",
      "C= 1000.0 : accuracy= 0.7760416666666666 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNX9//HXB2RxQUFAURBBxV3rEgH3vaJVKWItautSFf0K7hsqblg3XFB/opXihq2gsokVRSyopQoSBEWgCFqVCGJEUREj2+f3x7lphpBkJsudO0nez8djHszce2bmkyHJJ+ecez7H3B0REZGKNEg6ABERyX1KFiIikpaShYiIpKVkISIiaSlZiIhIWkoWIiKSlpKFiIikpWQhIiJpKVmIiEhaGyUdQE1p1aqVd+jQIekwRERqlRkzZnzj7q3TtaszyaJDhw7k5+cnHYaISK1iZp9n0k7DUCIikpaShYiIpKVkISIiadWZOQsRkfpu9erVFBQUUFRUtMG5pk2b0q5dOxo1alSl11ayEBGpIwoKCmjWrBkdOnTAzP533N1ZtmwZBQUFdOzYsUqvrWEoEZE6oqioiJYtW66XKADMjJYtW5bZ48iUkoWISB1SOlGkO54pDUOJxMgdXnoJVq+G44+HzTZLOiKRqom1Z2Fm3cxsvpktNLN+ZZxvb2aTzWymmX1oZieknLs+et58MzsuzjhF4rBmDfTpAz16wGmnQevW8NvfwrPPwvLlSUcnUjmxJQszawgMBo4HdgdON7PdSzXrD7zg7vsCvYBHo+fuHj3eA+gGPBq9nkitsGIFdO8Ojz0G114Lb70FvXvDjBlw1lmw1VZwwgnwxBPwzTdJRyt1ibtX6nim4uxZdAYWuvun7r4KGAF0L9XGgc2j+1sAi6P73YER7v6Lu/8XWBi9nkjOW7wYDjsMJkyAv/wF7rknPH7oIfj8c5g6FS67DP7zHzj/fGjTBo4+Gh59FJYsSTp6qc2aNm3KsmXLNkgMxVdDNW3atMqvHeecRVtgUcrjAqBLqTa3Aq+b2SXApsAxKc+dWuq5beMJU6TmzJ4degzLl8PLL4d5ilQNGkCXLuE2cCDMmgWjRoVbnz7Qty8cdBD07Blu7dsn83VI7dSuXTsKCgooLCzc4FzxOouqijNZlDX1XrofdDrwtLvfb2YHAs+a2Z4ZPhcz6w30BmivnypJ2Ouvw6mnwuabw5Qp8KtfVdzeDPbdN9z+/GeYO7ckcVx5ZbgdcEBJ4thpp+x8HVJ7NWrUqMrrKNKJcxiqANgu5XE7SoaZip0HvADg7u8CTYFWGT4Xdx/i7nnunte6ddoKuyKxGTo09Cg6dgzDTOkSRVl23x1uuin0NhYsgLvvDsf79YNOncJrDhgAc+aEq6xEsinOZDEd6GRmHc2sMWHCelypNl8ARwOY2W6EZFEYtetlZk3MrCPQCXgvxlhFqmTdOrjhBrjgAjj2WPjXv6AaPf3/2WknuO46eO+9MM8xaFDosdx6K+y5J+y2G/TvDzNnKnFIdsSWLNx9DdAXmADMI1z1NMfMBpjZyVGzq4ALzOwDYDhwjgdzCD2OucBrQB93XxtXrCJVUVQEZ54Jd90FF14Y5ig23zz98yqrfXu4/PKQiL78EgYPhrZtQ89jv/1gxx3hmmtCj2bdupp/fxEAq+7lVLkiLy/PtfmRZMuyZWHNxJQp4Wqna64JcxDZ9M03YcHfqFHwxhth4V/btnDKKWGO45BDoKEuOJc0zGyGu+elbadkIVI5CxeG+YkvvoBhw8KCu6QVX301alS4ZLeoKKzl6NEjJI4jjoAqFhuVOk7JQiQG77wDJ0eDqOPGhctcc82KFTB+fEgcr7wCP/0EW24Z4u7ZM8ytNGmSdJSSKzJNFiokKJKhF1+Eo44Kv3inTs3NRAGh/tRpp8Hzz0NhIYwdG3pCY8bASSeFsiNnnAGjR8PKlUlHK7WFkoVIGu5hAd1pp4V1D+++W3vWPGy8cSg78uyz8PXXocdx2mlhTUjPniFxnHoqDB8OP/yQdLSSyzQMJVKBNWvCqurHH4deveCpp6AaFRNyxpo18PbbYahq9Gj46ito3Bh+/euQRE4+OfSgpO7TnIVINf34Y/gr/LXX4PrrwyrrBnWwL75uXZiLKV49vmgRbLRRGHLr2TNc9bXVVklHKXFRshCphoICOPFE+OijUAzw/POTjig73CE/vyRxLFwYEuShh4bEccop4fJcqTuULESq6IMP4De/CWP4I0eGoZn6yD0URhw5MiSOuXPD8a5dS+pVxVSGSLJIV0OJVMGrr4bFbGZhwV19TRQQPoO99y6pRzVvXhiK++WXsAhxhx1g//3hzjth/vyko5W4KVmIRIYMCZeW7rQTTJsWflFKiV13hRtvhPffh08+CVeINWoUju26a6hZdcst8OGHqldVFylZSL23bl0o2nfhhXDcceEqoW23TTqq3LbDDiX1qL74Imzs1LIl3H57qI67887hooD8fCWOukJzFlKvFRWFbU5ffBH+7//g4YfDlUBSNUuXhkWAo0bBpEmwdi1sv31JvaoDD6ybV5TVZprgFknjm2/CgrV33oH77gubDWW7GGBdtmxZKIkyahRMnAirVsE225TUqzrsMCXmXKBkIVKBBQtCCYyCgrC6+dRTk46obvv++1CnatSocBHBzz9Dq1ZhDUfPnmFNR+PGSUdZPylZiJRjypTQo2jYMPzl27Vr0hHVLz/9FBLGqFHwj3+EwofNm4eLC3r2DFegbbxx0lHWH0oWImUYMQLOPhs6dAh1knbcMemI6reiojBENWpU2Jtj+XLYdFM48siQxLt0CfW4ttgi6UjrrkyThUYMpV5wDzvL3XBDWI08dqxqH+WCpk1Dj+Kkk8LmTZMnh8Tx1luh1wFhHmm33UqSR9eusMce2tgp29SzkDpv9Wq4+GIYOjSU5n7ySe3nUBt8913Yg3zatHCbOhW+/Tac23RTyMsrSSBduuhy56rSMJQIoWTH734XSnL37x9WI+uKp9rJPSwGnDq1JIHMnBkq6AJst11Jz6NLl7C6XHMf6SlZSL23aFGo8TRvXigx/qc/JR2R1LSiopAwihPI1Knw+efh3EYbhVX4qcNXnTrpj4XSlCykXps5MySKn34KY+DHHJN0RJItS5eWJI5p08JQ1ooV4VyLFtC5c0kC6dw5rDyvz5QspN4q3g1uyy3D/T33TDoiSdLataF3mZpAPvqopAxJp07rD1/tvXf9WvOhZCH10mOPhZ3t9tknXE2zzTZJRyS56McfQ92q4gQydWrokUC4Qmu//dafPG/fvu4OX+VEsjCzbsBDQENgqLvfXer8IODI6OEmwFbu3jw6NxD4DaHY4UTgMq8gWCWL+q24GOB994VNi4YPh802SzoqqS3cQ0HE1N7H+++HORGANm3W733k5UGzZsnGXFMSX2dhZg2BwcCxQAEw3czGufvc4jbufkVK+0uAfaP7BwEHA8VFoqcAhwNvxhWv1F4//wx//GOYm+jbFx58UNfgS+WYhYKH228fhjAh1LL68MP1E8hLL4VzDRqEtR6pCWS33er2912ci/I6Awvd/VMAMxsBdAfmltP+dOCW6L4DTYHGgAGNgKUxxiq11Ndfh9Id06bBoEFw2WV1d7hAsqtx49CDyMuDPn3CsWXLStZ+TJ0a/kAZOjSca9YsrDZPHb7aeuvk4q9pcSaLtsCilMcFQJeyGprZ9kBHYBKAu79rZpOBJYRk8Yi7zyvjeb2B3gDt27ev0eAl982fH4oBLlkSfmh79Eg6IqnrWraE448PNwjDnwsWrL9wcODAkrUfHTqs3/vYd98wJ1IbxZksyvr7rrw5h17ASHdfC2BmOwG7Ae2i8xPN7DB3f3u9F3MfAgyBMGdRI1FLrfD226FiaaNG8Oab4RJIkWxr0AB22SXczjorHFu5Msx3FCeQd96B558P5xo1ChdfpCaQHXesHb3hOJNFAbBdyuN2wOJy2vYC+qQ87gFMdfcVAGb2KtAVeLuM50o989xzcO65Ybe28eOhY8ekIxIpsckmYR/3Qw4pObZ4cUnymDYNnnoKHnkknGvZsmTYqmvX8IdP8+bJxF6ROJPFdKCTmXUEviQkhDNKNzKzXYAWwLsph78ALjCzuwg9lMOBB2OMVWoBd7jjDrjpJjjiCBg9OiyyEsl1224bhkmLh0rXrIG5c9dfef7qqyVrP3bddf3ex157Jb9RVGxv7+5rzKwvMIFw6eyT7j7HzAYA+e4+Lmp6OjCi1GWxI4GjgNmEoavX3P3luGKV3Ld6NVx0USgC+Mc/hknF+rRwSuqW4lIke+8NvXuHY99/H9Z+FCeQ8ePhmWfCuU02CbWuUhNIu3blv34ctChPct7334ed7N54A265JdxqwxivSHW4w2efbbj2Y9WqcL5t25Lhq4MPDreqSHydhUhN+PzzUONp/nx4+umwcZFIfWAW5uM6doRevcKxX36BDz5YP4GMHh0SxtSp8cajZCE5a8aMsBr7559hwoSwT7NIfdakSZgA79wZLrkkHCssDLe4NYj/LUQq7+WX4bDDwg/HO+8oUYiUp3Vr2H33+N9HyUJyziOPhDUUu+8eutbZ+EEQkYopWUjOWLsWrrwydK9POikstmvTJumoRAQ0ZyE5YuVK+MMfYMyYUN/p/vvrdlE2kdpGyUISt3QpnHwyTJ8ODz0El16adEQiUpqShSRq3rxQDHDp0tCr6N496YhEpCxKFpKYN98M5Q+aNIG33grlnUUkN2mCWxLx7LPw61+HmjlTpypRiOQ6JQvJKncYMCCUcz70UPj3v0PNfxHJbRqGkqxZtSoUTXvmmVC2Y8gQFQMUqS3Us5CsWL4cunULiWLAgFDPX4lCpPZQz0Ji99lnoRjgggUwbFgoMS4itYuShcQqPz8UA/zlF3j99bBpkYjUPhqGkti89BIcfjhsvHEoBqhEIVJ7KVlILB56KKyh2HPPcGnsbrslHZGIVIeShdSotWtDbafLLw+VYydPhq23TjoqEakuJQupMT/9BD17wsMPh+qxL74Y9g4WkdpPE9xSI776KpQVf//9sB9Fnz5JRyQiNUnJQqptzpxwaWxhYZjUPvHEpCMSkZqmYSiplkmT4OCDw6Wxb7+tRCFSV8WaLMysm5nNN7OFZtavjPODzGxWdPvYzJannGtvZq+b2Twzm2tmHeKMVSrvmWfguOOgXTuYNg323z/piEQkLrENQ5lZQ2AwcCxQAEw3s3HuPre4jbtfkdL+EmDflJcYBtzh7hPNbDNgXVyxSuW4w623hrIdxxwDI0fCFlskHZWIxCnOnkVnYKG7f+ruq4ARQEVb25wODAcws92Bjdx9IoC7r3D3lTHGKhlyh/POC4niT3+C8eOVKETqgziTRVtgUcrjgujYBsxse6AjMCk6tDOw3MxGm9lMM7s36qlIwsaMCUUAb7gBhg6FRo2SjkhEsiHOZGFlHPNy2vYCRrr72ujxRsChwNXAAcAOwDkbvIFZbzPLN7P8wsLC6kcsFSoqgquugr32gttuAyvrf1hE6qQ4k0UBsF3K43bA4nLa9iIagkp57sxoCGsNMBbYr/ST3H2Iu+e5e17r1q1rKGwpzwMPhAqyDz4IG+mia5F6Jc5kMR3oZGYdzawxISGMK93IzHYBWgDvlnpuCzMrzgBHAXNLP1ey58sv4c474ZRT4Kijko5GRLIttmQR9Qj6AhOAecAL7j7HzAaY2ckpTU8HRri7pzx3LWEI6p9mNpswpPXXuGKV9K6/HlavhnvvTToSEUmCpfyOrtXy8vI8Pz8/6TDqpGnToGvXkDDuvDPpaESkJpnZDHfPS9dOK7ilQuvWhSqy22wTkoWI1E+appQK/f3voWfxzDPQrFnS0YhIUtSzkHKtWAHXXQedO8Mf/pB0NCKSJPUspFx33QVLlsDo0dBAf1aI1Gv6FSBl+u9/4f77Q4+ia9ekoxGRpClZSJmuuQYaNoS77046EhHJBUoWsoHJk2HUqFD/qW2Z1bxEpL7JKFmY2Sgz+42ZKbnUcWvWwOWXQ4cOYR9tERHIvGfxGHAGsMDM7jazXWOMSRI0dCh8+GFYqb3xxklHIyK5IqNk4e5vuPuZhGJ+nwETzewdMzvXzFSkuo747jvo3x8OPxx69kw6GhHJJRkPK5lZS0KZ8POBmcBDhOQxMZbIJOsGDAgJ48EHVX5cRNaX0ToLMxsN7Ao8C5zk7kuiU8+bmQoy1QHz5sEjj8AFF8A++yQdjYjkmkwX5T3i7pPKOpFJASrJbe5wxRWw6aZw++1JRyMiuSjTYajdzKx58QMza2FmF8cUk2TZ+PEwYQLccgtoDykRKUumyeICd19e/MDdvwMuiCckyaZVq8IlsrvsAn36JB2NiOSqTIehGpiZFW9QZGYNgcbxhSXZ8sgj8PHHoXfRWP+jIlKOTJPFBOAFM/sL4MBFwGuxRSVZ8fXXcNttcMIJcPzxSUcjIrks02RxHXAh8H+ELU5fB4bGFZRkR//+sHIlPPBA0pGISK7LKFm4+zrCKu7H4g1HsmXmzLBa+4orwnyFiEhFMl1n0Qm4C9gdaFp83N13iCkuiZF72Cq1ZUu46aakoxGR2iDTq6GeIvQq1gBHAsMIC/SkFho5Ev71L7jjDmjePH17EZFMk8XG7v5PwNz9c3e/FTgqvrAkLj//DFdfDb/6FZx3XtLRiEhtkekEd1FUnnyBmfUFvgS2ii8sict998EXX8CwYWFzIxGRTGTas7gc2AS4FNgf+ANwdronmVk3M5tvZgvNrF8Z5weZ2azo9rGZLS91fnMz+9LMHskwTqnAokVhX+3f/S5UlhURyVTankW0AO80d78GWAGcm8kLR88bDBwLFADTzWycu88tbuPuV6S0vwTYt9TL3A68lcn7SXr9+sG6dTBwYNKRiEhtk7Zn4e5rgf3NKl20ujOw0N0/dfdVwAigewXtTweGFz8ws/2BrQlrOqSa3nkHnnsu7K3doUPS0YhIbZPpnMVM4CUzexH4qfigu4+u4DltgUUpjwuALmU1NLPtgY7ApOhxA+B+4I/A0eW9gZn1BnoDtG/fPpOvo15aty5cKtu2behdiIhUVqbJYktgGetfAeVARcmirJ6Il9O2FzAy6sUAXAyMd/dFFXVo3H0IMAQgLy+vvNeu94YNg/x8+NvfQhlyEZHKynQFd0bzFKUUANulPG4HLC6nbS8gtebpgcChURn0zYDGZrbC3fV3cSX98EPoTRx4IJxxRtLRiEhtlekK7qcoo1fg7n+q4GnTgU5m1pFwqW0vYINfV2a2C9ACeDfldc9MOX8OkKdEUTV33glLl8LLL2urVBGpukyHof6Rcr8p0IPyewkAuPuaaE3GBKAh8KS7zzGzAUC+u4+Lmp4OjCgufy4155NPYNAgOPtsOOCApKMRkdrMqvI7OpqAfsPdc2YVd15enufnazvwVD16wBtvhP0qttkm6WhEJBeZ2YxMtsfOtGdRWidAlx/lsDfegLFjwyI8JQoRqa5M5yx+ZP05i68Ie1xIDlqzBi6/HHbYIfwrIlJdmV4N1SzuQKTmPP44zJkDY8ZA06bp24uIpJNRbSgz62FmW6Q8bm5mv40vLKmqb7+Fm2+Go46C7hWtlxcRqYRMCwne4u7fFz9w9+XALfGEJNVx662wfDk8+KAulRWRmpNpsiirXVUnxyUmc+bAo4/CRRfBXnslHY2I1CWZJot8M3vAzHY0sx3MbBAwI87ApHLcw37am28OAwYkHY2I1DWZJotLgFXA88ALwM+sX55DEvbyyzBxItx2W9hbW0SkJlVpUV4uqs+L8n75BfbYAxo3hg8+gEaNko5IRGqLTBflZXo11EQza57yuIWZTahOgFJzHn64pLSHEoWIxCHTYahW0RVQALj7d2gP7pzw1Vdw++1w0klw3HFJRyMidVWmyWKdmf2vvIeZdaD8vSkki268EYqK4P77k45EROqyTC9/vRGYYmbF+2EfRrRDnSQnPx+eegquvho6dUo6GhGpyzIt9/GameUREsQs4CXCFVGSEPdQ96l1a+jfP+loRKSuy7SQ4PnAZYTd7mYBXQmbFeVMifL65vnn4d//hqFDw9oKEZE4ZTpncRlwAPC5ux8J7AsUxhaVVGjlSrjmGthvPzjnnKSjEZH6INM5iyJ3LzIzzKyJu/8n2g5VEjBwIBQUwPDh0LBh0tGISH2QabIoiNZZjAUmmtl3pNlWVeLxxRdwzz3QqxccckjS0YhIfZHpBHeP6O6tZjYZ2AJ4LbaopFzXXhuqyd5zT9KRiEh9UunKse7+VvpWEod//StMbN9yC7TXprYikkWZTnBLwtauhcsug3btQu9CRCSbtCdFLfH00zBzZpjU3mSTpKMRkfom1p6FmXUzs/lmttDM+pVxfpCZzYpuH5vZ8uj4Pmb2rpnNMbMPzez3ccaZ677/Hm64AQ4+GH5frz8JEUlKbD0LM2sIDAaOBQqA6WY2zt3nFrdx9ytS2l9CWL8BsBI4y90XmNm2wAwzm5BazLA++fOfobAQxo/XVqkikow4exadgYXu/qm7rwJGAN0raH86MBzA3T929wXR/cXA10DrGGPNWQsWwEMPwbnnwv77Jx2NiNRXcSaLtsCilMcF0bENmNn2QEdgUhnnOgONgU9iiDHnXXUVNG0Kd9yRdCQiUp/FmSzKGjApr6x5L2Cku69d7wXMtgGeBc5193UbvIFZbzPLN7P8wsK6V31kwoSwXepNN0GbNklHIyL1WZzJogDYLuVxO8pf9d2LaAiqmJltDrwC9Hf3qWU9yd2HuHueu+e1bl23RqlWr4YrroCddoJLL006GhGp7+K8dHY60MnMOgJfEhLCGaUbRTWmWhCq2BYfawyMAYa5+4sxxpizHnsM5s2DceOgSZOkoxGR+i62noW7rwH6AhOAecAL7j7HzAaY2ckpTU8HRrh76hDVaYQNls5JubR2n7hizTXffBNWaR97LJx4YtLRiIiArf87uvbKy8vz/Pz8pMOoEX36wOOPwwcfwB57JB2NiNRlZjbD3fPStVO5jxwzezb85S9w8cVKFCKSO5Qscoh7qP/UvDncemvS0YiIlFBtqBwydixMngyDB8OWWyYdjYhICfUsckRRUViAt+ee0Lt30tGIiKxPPYsc8eCD8N//whtvwEb6XxGRHKOeRQ5YvDgUC/ztb+Hoo5OORkRkQ0oWOeCGG8KK7fvuSzoSEZGyKVkk7L334Jln4MorYccdk45GRKRsShYJKr5Utk2b0LsQEclVmkpN0HPPwdSp8NRT0KxZ0tGIiJRPPYuErFgB114LeXlw1llJRyMiUjH1LBJyzz3hKqiRI6GBUraI5Dj9mkrAZ5/BvffCmWfCgQcmHY2ISHpKFgm45hpo2BDuvjvpSEREMqNkkWVvvRWGnvr1g3btko5GRCQzShZZtHZtuFS2fXu4+uqkoxERyZwmuLPoiSfChkYvvAAbb5x0NCIimVPPIkuWL4cbb4TDDoNTT006GhGRylGyyJIBA2DZMnjoITBLOhoRkcpRssiC//wH/t//g/PPh332SToaEZHKU7LIgquugk02CWXIRURqI01wx2z8+HC7/37YaqukoxERqRr1LGK0alUoPb7zztC3b9LRiIhUXazJwsy6mdl8M1toZv3KOD/IzGZFt4/NbHnKubPNbEF0OzvOOOMyeDDMnw+DBkHjxklHIyJSdbENQ5lZQ2AwcCxQAEw3s3HuPre4jbtfkdL+EmDf6P6WwC1AHuDAjOi538UVb00rLITbboPjj4cTTkg6GhGR6omzZ9EZWOjun7r7KmAE0L2C9qcDw6P7xwET3f3bKEFMBLrFGGuNu+km+OkneOCBpCMREam+OJNFW2BRyuOC6NgGzGx7oCMwqbLPzUWzZsGQIWGeYtddk45GRKT64kwWZS0983La9gJGuvvayjzXzHqbWb6Z5RcWFlYxzJrlDpdfDi1bws03Jx2NiEjNiDNZFADbpTxuBywup20vSoagMn6uuw9x9zx3z2vdunU1w60Zo0aFyrJ//jO0aJF0NCIiNSPOZDEd6GRmHc2sMSEhjCvdyMx2AVoA76YcngD82sxamFkL4NfRsZz288+hmuzee4fV2iIidUVsV0O5+xoz60v4Jd8QeNLd55jZACDf3YsTx+nACHf3lOd+a2a3ExIOwAB3/zauWGvKAw/A55/D5MlhcyMRkbrCUn5H12p5eXmen5+f2Pt/+WVYfHf88WFzIxGR2sDMZrh7Xrp2WsFdQ/r1C5sb3Xtv0pGIiNQ8JYsa8O678Le/hfmKjh2TjkZEpOYpWVTTunVhq9Rttw29CxGRukhVZ6vp2Wdh+nQYNgw22yzpaERE4qGeRTX8+CNcfz106QJnnpl0NCIi8VHPohruuguWLIExY6CB0q6I1GH6FVdFn34aNjQ666zQsxARqcuULKro6quhUaPQuxARqes0DFUFkyaFoac77wxXQYmI1HXqWVTSmjWhqmzHjnDFFenbi4jUBepZVNJf/wqzZ4fqsk2bJh2NiEh2qGdRCd9+G3bAO/JI6NEj6WhERLJHyaISbrsNvvsOHnwQrKztmURE6igliwzNnQuDB8OFF4b9KkRE6hMliwy4h8nsZs1gwICkoxERyT5NcGfglVfg9dfD8FOrVklHIyKSfepZpLFqVehV7LorXHxx0tGIiCRDPYs0Hn4YFi6E114LK7ZFROoj9SwqsHRpmKM48UQ47rikoxERSY6SRQX694eiolAwUESkPlOyKMf778MTT8Cll8LOOycdjYhIspQsyuAetkpt1Sqs2BYRqe80wV2GF16AKVNCHagttkg6GhGR5MXaszCzbmY238wWmlm/ctqcZmZzzWyOmT2XcnxgdGyemT1slp0CGytXwjXXwL77wrnnZuMdRURyX2w9CzNrCAwGjgUKgOlmNs7d56a06QRcDxzs7t+Z2VbR8YOAg4HiwhpTgMOBN+OKt9i998KiRfD3v0PDhnG/m4hI7RBnz6IzsNDdP3X3VcAIoHupNhcAg939OwB3/zo67kBToDHQBGgELI0xViAkiXvugdNOg0MPjfvdRERqjziTRVtgUcrjguhYqp2Bnc3s32Y21cy6Abj7u8BkYEmSOi+OAAAIT0lEQVR0m+Du80q/gZn1NrN8M8svLCysdsDXXRcmtwcOrPZLiYjUKXEmi7LmGLzU442ATsARwOnAUDNrbmY7AbsB7QgJ5igzO2yDF3Mf4u557p7XunXragU7ZQoMHw7XXgvbb1+tlxIRqXPiTBYFwHYpj9sBi8to85K7r3b3/wLzCcmjBzDV3Ve4+wrgVaBrXIGuWxculW3XLiQLERFZX5zJYjrQycw6mlljoBcwrlSbscCRAGbWijAs9SnwBXC4mW1kZo0Ik9sbDEPVlKefDovwBg6ETTeN611ERGqv2JKFu68B+gITCL/oX3D3OWY2wMxOjppNAJaZ2VzCHMU17r4MGAl8AswGPgA+cPeX44jzhx/ghhvgoIOgV6843kFEpPaLdVGeu48Hxpc6dnPKfQeujG6pbdYCF8YZW7GVK+HAA+HGG7VVqohIeer9Cu42bWDMmKSjEBHJbaoNJSIiaSlZiIhIWkoWIiKSlpKFiIikpWQhIiJpKVmIiEhaShYiIpKWkoWIiKRlYRF17WdmhcDn1XiJVsA3NRROTVJclaO4KkdxVU5djGt7d09btrvOJIvqMrN8d89LOo7SFFflKK7KUVyVU5/j0jCUiIikpWQhIiJpKVmUGJJ0AOVQXJWjuCpHcVVOvY1LcxYiIpKWehYiIpJWvU0WZvY7M5tjZuvMrNyrCMysm5nNN7OFZtYvC3FtaWYTzWxB9G+LctqtNbNZ0a30drU1GU+FX7+ZNTGz56Pz08ysQ1yxVCKmc8ysMOXzOT/umKL3fdLMvjazj8o5b2b2cBT3h2a2X47EdYSZfZ/yed1cVrsY4trOzCab2bzoZ/GyMtpk/TPLMK6sf2Zm1tTM3jOzD6K4biujTXw/j+5eL2/AbsAuwJtAXjltGhK2d90BaEzY4nX3mOMaCPSL7vcD7imn3YosfEZpv37gYuAv0f1ewPM5ENM5wCMJfE8dBuwHfFTO+ROAVwEDugLTciSuI4B/JPB5bQPsF91vBnxcxv9l1j+zDOPK+mcWfQabRfcbAdOArqXaxPbzWG97Fu4+z93np2nWGVjo7p+6+ypgBNA95tC6A89E958Bfhvz+1Ukk68/Nd6RwNFmsW5Qm8T/SUbc/W3g2wqadAeGeTAVaG5m2+RAXIlw9yXu/n50/0dgHtC2VLOsf2YZxpV10WewInrYKLqVnnSO7eex3iaLDLUFFqU8LiD+b5qt3X0JhG9aYKty2jU1s3wzm2pmcSWUTL7+/7Vx9zXA90DLmOLJNCaAntGwxUgz2y7GeCojie+nTB0YDW+8amZ7ZPvNo+GSfQl/LadK9DOrIC5I4DMzs4ZmNgv4Gpjo7uV+XjX981in9+A2szeANmWcutHdX8rkJco4Vu3LxyqKqxIv097dF5vZDsAkM5vt7p9UN7ZSMvn6Y/mMKpDJ+70MDHf3X8zsIsJfWkfFGFOmsv1ZZep9QsmHFWZ2AjAW6JStNzezzYBRwOXu/kPp02U8JSufWZq4EvnM3H0tsI+ZNQfGmNme7p46FxXb51Wnk4W7H1PNlygAUv8qbQcsruZrVhiXmS01s23cfUnU3f66nNdYHP37qZm9Sfjrp6aTRSZff3GbAjPbCNiCeIc80sbk7stSHv4VuCfGeCojlu+n6kr9Reju483sUTNr5e6x10Ays0aEX8h/d/fRZTRJ5DNLF1eSn1n0nsujn/tuQGqyiO3nUcNQFZsOdDKzjmbWmDBhFNuVR5FxwNnR/bOBDXpAZtbCzJpE91sBBwNzY4glk68/Nd5TgUkeza7FJG1Mpca0TyaMOeeCccBZ0RU+XYHvi4cck2RmbYrHtc2sM+H3wrKKn1Uj72vAE8A8d3+gnGZZ/8wyiSuJz8zMWkc9CsxsY+AY4D+lmsX385jN2fxcugE9CFn4F2ApMCE6vi0wPqXdCYSrIT4hDF/FHVdL4J/AgujfLaPjecDQ6P5BwGzClUCzgfNijGeDrx8YAJwc3W8KvAgsBN4DdsjCZ5QupruAOdHnMxnYNUvfU8OBJcDq6HvrPOAi4KLovAGDo7hnU85VeAnE1Tfl85oKHJSluA4hDJF8CMyKbick/ZllGFfWPzNgb2BmFNdHwM3R8az8PGoFt4iIpKVhKBERSUvJQkRE0lKyEBGRtJQsREQkLSULERFJS8lCpBLMbEX6VhU+f2S06h4z28zMHjezT6Iqom+bWRczaxzdr9OLZqV2UbIQyZKoflBDd/80OjSUsLq2k7vvQaiW28pDgcR/Ar9PJFCRMihZiFRBtKL4XjP7yMxmm9nvo+MNotIPc8zsH2Y23sxOjZ52JtGKfDPbEegC9Hf3dRBKt7j7K1HbsVF7kZygbq5I1ZwC7AP8CmgFTDeztwmlVzoAexEqBs8DnoyeczBhNTXAHsAsD4XhyvIRcEAskYtUgXoWIlVzCKGy7Vp3Xwq8Rfjlfgjworuvc/evCOVGim0DFGby4lESWWVmzWo4bpEqUbIQqZryNpSpaKOZnwm1eyDUFfqVmVX0M9gEKKpCbCI1TslCpGreBn4fbUbTmrB16XvAFMLGSw3MbGvC9pvF5gE7AXjYeyQfuC2lemknM+se3W8JFLr76mx9QSIVUbIQqZoxhOqfHwCTgGujYadRhMquHwGPE3ZY+z56ziusnzzOJ2yCtdDMZhP23ijeq+FIYHy8X4JI5lR1VqSGmdlmHnZQa0nobRzs7l9FexBMjh6XN7Fd/Bqjges9/T7xIlmhq6FEat4/ok1qGgO3Rz0O3P1nM7uFsE/yF+U9OdrUaawSheQS9SxERCQtzVmIiEhaShYiIpKWkoWIiKSlZCEiImkpWYiISFpKFiIiktb/B8R2iRWVmVwXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "C_s = np.logspace(-1, 3, 5)# logspace(a,b,N)把10的a次方到10的b次方区间分成N份  \n",
    "\n",
    "\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "\n",
    "    tmp = fit_grid_point(oneC, X_train_part, y_train_part, X_val, y_val)\n",
    "    accuracy_s.append(tmp)\n",
    "\n",
    "x_axis = np.log10(C_s)\n",
    "\n",
    "plt.plot(x_axis, np.array(accuracy_s), 'b-')\n",
    "    \n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'accuracy' )\n",
    "\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "随着C的增大，函数成凸函数趋势，当C为10时，准确率最高。\n",
    "保持C不变时，分别尝试了gamma=1,100,1000,其准确率变化不大，但是对交叉验证结果影响很大，gamma增大，logloss减小"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10.0\n"
     ]
    }
   ],
   "source": [
    "### 最佳超参数\n",
    "index = np.argmax(accuracy_s, axis=None)\n",
    "Best_C2 = C_s[ index ]\n",
    "\n",
    "print(Best_C2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "Best_C = 100\n",
    "\n",
    "SVC6 = LinearSVC(C = Best_C)\n",
    "SVC6.fit(X_train, y_train)\n",
    "\n",
    "#保持模型，用于后续测试\n",
    "import pickle\n",
    "pickle.dump(SVC3, open(\"Otto_LinearSVC.pkl\", 'wb'))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
